Using output codes to boost multiclass learning problems
نویسنده
چکیده
This paper describes a new technique for solving multiclass learning problems by combining Freund and Schapire’s boosting algorithm with the main ideas of Dietterich and Bakiri’s method of error-correcting output codes (ECOC). Boosting is a general method of improving the accuracy of a given base or “weak” learning algorithm. ECOC is a robust method of solving multiclass learning problems by reducing to a sequence of two-class problems. We show that our new hybrid method has advantages of both: Like ECOC, our method only requires that the base learning algorithm work on binary-labeled data. Like boosting, we prove that the method comes with strong theoretical guarantees on the training and generalization error of the final combined hypothesis assuming only that the base learning algorithm perform slightly better than random guessing. Althoughprevious methods were known for boostingmulticlass problems, the new method may be significantly faster and require less programming effort in creating the base learning algorithm. We also compare the new algorithm experimentally to other voting methods.
منابع مشابه
Effectiveness of Error Correcting Output Codes in Multiclass Learning Problems
Classification (machine learning): How does one algorithmically classify the though a more effective approach could be using error correcting codes: @(cs/9501101) Solving Multiclass Learning Problems via Error-Correcting Output Codes. to solving machine learning problems can be broadly useful.
متن کاملMulticlass Learning by Probabilistic Embeddings
We describe a new algorithmic framework for learning multiclass categorization problems. In this framework a multiclass predictor is composed of a pair of embeddings that map both instances and labels into a common space. In this space each instance is assigned the label it is nearest to. We outline and analyze an algorithm, termed Bunching, for learning the pair of embeddings from labeled data...
متن کاملError-Correcting Output Codes: A General Method for Improving Multiclass Inductive Learning Programs
Multiclass learning problems involve nding a deeni-tion for an unknown function f (x) whose range is a discrete set containing k > 2 values (i.e., k \classes"). The deenition is acquired by studying large collections of training examples of the form hx i ; f (x i)i. Existing approaches to this problem include (a) direct application of multiclass algorithms such as the decision-tree algorithms I...
متن کاملSolving Multiclass Learning Problems via Error-Correcting Output Codes
Multiclass learning problems involve nding a de nition for an unknown function f(x) whose range is a discrete set containing k > 2 values (i.e., k \classes"). The de nition is acquired by studying collections of training examples of the form hxi; f(xi)i. Existing approaches to multiclass learning problems include direct application of multiclass algorithms such as the decision-tree algorithms C...
متن کاملSolving Multiclass Learning Problems viaError - Correcting Output
Multiclass learning problems involve nding a deenition for an unknown function f (x) whose range is a discrete set containing k > 2 values (i.e., k \classes"). The deenition is acquired by studying collections of training examples of the form hx i ; f (x i)i. Existing approaches to multiclass learning problems include direct application of multiclass algorithms such as the decision-tree algorit...
متن کامل